fix: align custom tool SSE lifecycle with OpenAI API compatibility - #158

Function-call SSE translation refactor

• Detecting normalized function calls that represent gateway-owned tools.
• Hiding their internal function_call events.
• Buffering calls whose names were unavailable in output_item.added.
• Flushing ordinary function events once their ownership was known.
• Deferring events after a gateway-owned call to preserve output ordering.
• Synthesizing the public custom-tool lifecycle after inference completed.

This logic was spread across emit_upstream_stream_event, defer_after_gateway_call, record_first_hidden_gateway_output_index, defer_or_flush_function_event, should_hide_upstream_event, flush_pending_function_events, and drop_pending_function_events in executor/upstream.rs.

The refactor moves function-call classification and public wire translation into a dedicated FunctionSseTranslator. The translator receives only a request-scoped name → ToolType map, rather than the full ToolRegistry, and handles normalized function calls according to their declared public type:

• Ordinary functions and namespace members pass through unchanged.
• Custom tools are translated incrementally from function_call_arguments.* into custom_tool_call_input.*.
• Gateway-owned MCP and web-search function frames are suppressed until their execution-specific public lifecycle is emitted.
• Calls without an initial name are buffered only until their type can be resolved, with a 256 KiB safety limit.

ResponseAccumulator still processes the original upstream frame first. This keeps the normalized FunctionToolCall as the authoritative internal representation used for dispatch and model continuation. The translator then reads that accumulated call state to produce client-facing frames, avoiding a second argument accumulator or changes to internal output items.

pending_unnamed is keyed only by the original item_id, so when output_item.done supplies a new stable ID we lose the buffered output_item.added and argument deltas. i reproduced this and only output_item.done reached the client. we should resolve pending calls by item_id or output_index and add this case as a regression test.

Pending calls are now resolved by either the original item_id or output_index, with a regression test confirming buffered added and delta frames survive when output_item.done supplies a new stable ID.

an unnamed function call doesn’t establish a defer boundary while we wait to classify it. this lets later output items reach the client before an earlier custom or gateway-owned call, leaving the SSE lifecycle out of order. we should establish a provisional boundary at the unnamed call’s output_index and release it once the call is classified.

Unnamed calls now establish a provisional defer boundary at their output_index; it is released and queued frames are flushed after client-owned classification, or retained when the call is gateway-owned.

the Lark or regex grammar is only copied into the function description while the generated input schema still accepts any string. that doesn’t preserve the constrained-decoding behavior of an OpenAI custom tool grammar. we should enforce or validate the grammar, or reject grammar formats we can’t support instead of treating them as prompt guidance. OpenAI’s grammar documentation

Custom-tool grammar formats are no longer copied into the function description as prompt guidance. CustomHandler::validate now rejects any format declaration because function normalization cannot preserve OpenAI’s constrained-decoding semantics. RequestPayload::to_upstream_request() invokes ResponsesTool::validate() before normalization, and regression tests cover both Lark and regex formats. Unformatted custom tools remain supported.

normalized_for_upstream rewrites every custom selector as a function selector without checking the corresponding declaration. a request with a function named echo and tool_choice: {"type":"custom","name":"echo"} is therefore accepted as a function choice instead of rejected. we should validate selectors against the declaration map before normalizing them.

Custom selectors are now validated against the request’s custom-tool declaration map before normalization, so a custom selector cannot silently target a function with the same name; valid custom selectors and allowed_tools entries are rewritten to functions only after validation.